A Powerful Feature Selection approach based on Mutual Information

Feature selection aims to reduce the dimensionality of patterns for classificatory analysis by selecting the most informative instead of irrelevant and/or redundant features. In this paper we propose a novel feature selection measure based on mutual information and takes into consideration the interaction between features. The proposed measure is used to determine relevant features from the original feature set for a pattern recognition problem. We use a Support Vector Machine (SVM) classifier to compare the performance of our measure with recently proposed information theoretic criteria. Very good performances are obtained when applying this method on handwritten digital recognition data.